Fast Network Pruning and Feature Extraction by using the Unit-OBS Algorithm
نویسندگان
چکیده
The algorithm described in this article is based on the OBS algorithm by Hassibi, Stork and Wolff ([1] and [2]). The main disadvantage of OBS is its high complexity. OBS needs to calculate the inverse Hessian to delete only one weight (thus needing much time to prune a big net) . A better algorithm should use this matrix to remove more than only one weight , because calculating the inverse Hessian takes the most time in the OBS algorithm. The algorithm, called UnitOBS, described in this article is a method to overcome this disadvantage. This algorithm only needs to calculate the inverse Hessian once to remove one whole unit thus drastically reducing the time to prune big nets. A further advantage of UnitOBS is that it can be used to do a feature extraction on the input data. This can be helpful on the understanding of unknown problems.
منابع مشابه
Fast Network Pruning and FeatureExtraction Using the Unit - OBS
The algorithm described in this article is based on the OBS algorithm by Hassibi, Stork and Woll ((1] and 2]). The main disadvantage of OBS is its high complexity. OBS needs to calculate the inverse Hessian to delete only one weight (thus needing much time to prune a big net). A better algorithm should use this matrix to remove more than only one weight, because calculating the inverse Hessian ...
متن کاملFast Voltage and Power Flow Contingency Ranking Using Enhanced Radial Basis Function Neural Network
Deregulation of power system in recent years has changed static security assessment to the major concerns for which fast and accurate evaluation methodology is needed. Contingencies related to voltage violations and power line overloading have been responsible for power system collapse. This paper presents an enhanced radial basis function neural network (RBFNN) approach for on-line ranking of ...
متن کاملA Comparative Study of Neural Network Optimization Techniques
In the last years we developed ENZO, an evolutionary neural network optimizer which surpasses other algorithms with regard to performance and scalability. In this study we compare ENZO to standard techniques for topology optimization: Optimal Brain Surgeon (OBS), Magnitude based Pruning (MbP), and to an improved algorithm deduced from OBS (unit-OBS). Furthermore we compare results to a newly pr...
متن کاملComparative Analysis of Wavelet-based Feature Extraction for Intramuscular EMG Signal Decomposition
Background: Electromyographic (EMG) signal decomposition is the process by which an EMG signal is decomposed into its constituent motor unit potential trains (MUPTs). A major step in EMG decomposition is feature extraction in which each detected motor unit potential (MUP) is represented by a feature vector. As with any other pattern recognition system, feature extraction has a significant impac...
متن کاملFeature Selection in Structural Health Monitoring Big Data Using a Meta-Heuristic Optimization Algorithm
This paper focuses on the processing of structural health monitoring (SHM) big data. Extracted features of a structure are reduced using an optimization algorithm to find a minimal subset of salient features by removing noisy, irrelevant and redundant data. The PSO-Harmony algorithm is introduced for feature selection to enhance the capability of the proposed method for processing the measure...
متن کامل